2026-04-10 18:22:29,081 [ 268640 ] INFO : ClickHouse root is not set. Will use /home/ubuntu/_work/ClickHouse/ClickHouse (runner:53, check_args_and_update_paths) 2026-04-10 18:22:29,081 [ 268640 ] INFO : Cases dir is not set. Will use /home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration (runner:79, check_args_and_update_paths) 2026-04-10 18:22:29,081 [ 268640 ] INFO : utils dir is not set. Will use /home/ubuntu/_work/ClickHouse/ClickHouse/utils (runner:90, check_args_and_update_paths) 2026-04-10 18:22:29,081 [ 268640 ] INFO : base_configs_dir: /home/ubuntu/_work/ClickHouse/ClickHouse/programs/server, binary: /home/ubuntu/_work/_temp/test/build/clickhouse, cases_dir: /home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration (runner:92, check_args_and_update_paths) clickhouse_integration_tests_volume Running pytest container as: 'docker run --rm --name clickhouse_integration_tests_q1p96k --privileged --dns-search='.' --memory=30709039104 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --volume=/home/ubuntu/_work/_temp/test/build/clickhouse:/clickhouse --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/programs/server:/clickhouse-config --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration:/ClickHouse/tests/integration --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/backupview:/ClickHouse/utils/backupview --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/grpc-client/pb2:/ClickHouse/utils/grpc-client/pb2 --volume=/run:/run/host:ro --volume=clickhouse_integration_tests_volume:/var/lib/docker -e DOCKER_DOTNET_CLIENT_TAG=11de0b29a15d -e DOCKER_HELPER_TAG=5dc43a6382f0 -e DOCKER_BASE_TAG=5ccda723c1fc -e DOCKER_KERBEROS_KDC_TAG=9391ecdee8d7 -e DOCKER_MYSQL_GOLANG_CLIENT_TAG=9bec2a638e6e -e DOCKER_MYSQL_JAVA_CLIENT_TAG=766bff31cfe4 -e DOCKER_MYSQL_JS_CLIENT_TAG=41ba7c2ec2a1 -e DOCKER_MYSQL_PHP_CLIENT_TAG=88be89c1e3b6 -e DOCKER_NGINX_DAV_TAG=b55ac9cd7519 -e DOCKER_POSTGRESQL_JAVA_CLIENT_TAG=a4eff5c7f4d6 -e DOCKER_PYTHON_BOTTLE_TAG=d862517635bf -e DOCKER_CLIENT_TIMEOUT=300 -e COMPOSE_HTTP_TIMEOUT=600 -e CLICKHOUSE_USE_OLD_ANALYZER=1 -e PYTHONUNBUFFERED=1 -e PYTEST_ADDOPTS="--dist=loadfile -n 10 -rfEps --run-id=0 --color=no --durations=0 --report-log=parallel1_0.jsonl --report-log-exclude-logs-on-passed-tests test_executable_dictionary/test.py::test_executable_input_bash test_executable_dictionary/test.py::test_executable_input_python test_executable_dictionary/test.py::test_executable_input_send_chunk_header_python test_executable_dictionary/test.py::test_executable_input_signalled_python test_executable_dictionary/test.py::test_executable_input_slow_python test_executable_dictionary/test.py::test_executable_input_sum_python test_executable_dictionary/test.py::test_executable_non_direct_input_bash test_executable_dictionary/test.py::test_executable_source_argument_python test_executable_dictionary/test.py::test_executable_source_exit_code_check test_executable_dictionary/test.py::test_executable_source_python test_executable_dictionary/test.py::test_executable_source_updated_python test_format_avro_confluent/test.py::test_select test_format_avro_confluent/test.py::test_select_auth test_format_avro_confluent/test.py::test_select_auth_encoded test_format_avro_confluent/test.py::test_select_auth_encoded_complex test_grant_and_revoke/test_without_table_engine_grant.py::test_table_engine_and_source_grant test_groupBitmapAnd_on_distributed/test_groupBitmapAndState_on_distributed_table.py::test_groupBitmapAndState_on_distributed_table test_keeper_profiler/test.py::test_profiler test_keeper_reconfig_remove_many/test.py::test_reconfig_remove_2_and_leader test_keeper_restore_from_snapshot/test_disk_s3.py::test_recover_from_snapshot_with_disk_s3 test_keeper_snapshot_small_distance/test.py::test_snapshot_and_load test_kerberos_auth/test.py::test_bad_path_to_keytab test_kerberos_auth/test.py::test_kerberos_auth_with_keytab test_kerberos_auth/test.py::test_kerberos_auth_without_keytab test_log_levels_update/test.py::test_log_levels_update test_log_query_probability/test.py::test_log_quries_probability_one test_log_query_probability/test.py::test_log_quries_probability_two test_match_process_uid_against_data_owner/test.py::test_different_user test_max_authentication_methods_per_user/test.py::test_alter test_max_authentication_methods_per_user/test.py::test_alter_default_setting test_max_authentication_methods_per_user/test.py::test_create test_max_authentication_methods_per_user/test.py::test_create_default_setting test_merge_tree_check_part_with_cache/test.py::test_check_part_with_cache 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_multiple_shards_multiple_replicas-custom_key_sampling-key]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_multiple_shards_multiple_replicas-custom_key_sampling-sipHash64(key)]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_single_shard_multiple_replicas-custom_key_sampling-key]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_range-key]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_range-sipHash64(key)]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_sampling-key]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_range-key]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_range-sipHash64(key)]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_sampling-key]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)]' test_parallel_replicas_increase_error_count/test.py::test_increase_error_count 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-1000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-10000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-100000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-1000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-10000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-100000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-1000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-10000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-100000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-1000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-10000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-100000-SELECT sum(key) FROM {table_name}]' test_password_constraints/test.py::test_complexity_rules test_password_constraints/test.py::test_default_password_type test_polymorphic_parts/test.py::test_compact_parts_only 'test_polymorphic_parts/test.py::test_different_part_types_on_replicas[polymorphic_table_compact-Compact]' 'test_polymorphic_parts/test.py::test_different_part_types_on_replicas[polymorphic_table_wide-Wide]' 'test_polymorphic_parts/test.py::test_polymorphic_parts_basics[first_node0-second_node0]' 'test_polymorphic_parts/test.py::test_polymorphic_parts_basics[first_node1-second_node1]' test_polymorphic_parts/test.py::test_polymorphic_parts_index test_polymorphic_parts/test.py::test_polymorphic_parts_non_adaptive test_profile_events_s3/test.py::test_profile_events 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-False-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-False-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-True-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-True-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-False-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-False-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-True-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-True-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-False-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-False-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-True-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-True-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-False-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-False-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-True-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-True-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-False-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-False-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-True-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-True-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-False-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-False-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-True-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-True-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-False-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-False-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-True-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-True-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-False-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-False-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-True-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-True-True]' -vvv " altinityinfra/integration-tests-runner:226bfaf75ac1 '. Start tests ============================= test session starts ============================== platform linux -- Python 3.10.12, pytest-7.4.4, pluggy-1.5.0 -- /usr/bin/python3 cachedir: .pytest_cache Test order randomisation NOT enabled. Enable with --random-order or --random-order-bucket= rootdir: /ClickHouse/tests/integration configfile: pytest.ini plugins: timeout-2.3.1, repeat-0.9.3, order-1.0.0, reportlog-0.4.0, xdist-3.5.0, random-order-1.1.1 timeout: 900.0s timeout method: signal timeout func_only: False created: 10/10 workers 10 workers [100 items] scheduling tests via LoadFileScheduling test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_multiple_shards_multiple_replicas-custom_key_sampling-key] test_executable_dictionary/test.py::test_executable_input_bash test_polymorphic_parts/test.py::test_compact_parts_only test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-1000-SELECT sum(key) FROM {table_name}] test_password_constraints/test.py::test_complexity_rules test_kerberos_auth/test.py::test_bad_path_to_keytab test_log_query_probability/test.py::test_log_quries_probability_one test_max_authentication_methods_per_user/test.py::test_alter test_format_avro_confluent/test.py::test_select test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-False-False] [gw6] [ 1%] PASSED test_max_authentication_methods_per_user/test.py::test_alter test_max_authentication_methods_per_user/test.py::test_alter_default_setting [gw9] [ 2%] PASSED test_kerberos_auth/test.py::test_bad_path_to_keytab test_kerberos_auth/test.py::test_kerberos_auth_with_keytab [gw6] [ 3%] PASSED test_max_authentication_methods_per_user/test.py::test_alter_default_setting test_max_authentication_methods_per_user/test.py::test_create [gw9] [ 4%] PASSED test_kerberos_auth/test.py::test_kerberos_auth_with_keytab test_kerberos_auth/test.py::test_kerberos_auth_without_keytab [gw8] [ 5%] PASSED test_password_constraints/test.py::test_complexity_rules test_password_constraints/test.py::test_default_password_type [gw9] [ 6%] PASSED test_kerberos_auth/test.py::test_kerberos_auth_without_keytab [gw8] [ 7%] PASSED test_password_constraints/test.py::test_default_password_type [gw6] [ 8%] PASSED test_max_authentication_methods_per_user/test.py::test_create test_max_authentication_methods_per_user/test.py::test_create_default_setting [gw6] [ 9%] PASSED test_max_authentication_methods_per_user/test.py::test_create_default_setting [gw2] [ 10%] PASSED test_executable_dictionary/test.py::test_executable_input_bash test_executable_dictionary/test.py::test_executable_input_python [gw2] [ 11%] PASSED test_executable_dictionary/test.py::test_executable_input_python test_executable_dictionary/test.py::test_executable_input_send_chunk_header_python [gw1] [ 12%] PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_multiple_shards_multiple_replicas-custom_key_sampling-key] test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_multiple_shards_multiple_replicas-custom_key_sampling-sipHash64(key)] [gw2] [ 13%] PASSED test_executable_dictionary/test.py::test_executable_input_send_chunk_header_python test_executable_dictionary/test.py::test_executable_input_signalled_python test_groupBitmapAnd_on_distributed/test_groupBitmapAndState_on_distributed_table.py::test_groupBitmapAndState_on_distributed_table [gw2] [ 14%] PASSED test_executable_dictionary/test.py::test_executable_input_signalled_python test_executable_dictionary/test.py::test_executable_input_slow_python test_keeper_reconfig_remove_many/test.py::test_reconfig_remove_2_and_leader [gw1] [ 15%] PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_multiple_shards_multiple_replicas-custom_key_sampling-sipHash64(key)] test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_single_shard_multiple_replicas-custom_key_sampling-key] [gw1] [ 16%] PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_single_shard_multiple_replicas-custom_key_sampling-key] test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)] [gw2] [ 17%] PASSED test_executable_dictionary/test.py::test_executable_input_slow_python test_executable_dictionary/test.py::test_executable_input_sum_python [gw2] [ 18%] PASSED test_executable_dictionary/test.py::test_executable_input_sum_python test_executable_dictionary/test.py::test_executable_non_direct_input_bash [gw1] [ 19%] PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)] test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_range-key] [gw2] [ 20%] PASSED test_executable_dictionary/test.py::test_executable_non_direct_input_bash test_executable_dictionary/test.py::test_executable_source_argument_python [gw0] [ 21%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-False-False] [gw5] [ 22%] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-1000-SELECT sum(key) FROM {table_name}] test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-10000-SELECT sum(key) FROM {table_name}] test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-False-True] test_keeper_profiler/test.py::test_profiler [gw2] [ 23%] PASSED test_executable_dictionary/test.py::test_executable_source_argument_python test_executable_dictionary/test.py::test_executable_source_exit_code_check [gw1] [ 24%] PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_range-key] test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_range-sipHash64(key)] [gw2] [ 25%] PASSED test_executable_dictionary/test.py::test_executable_source_exit_code_check test_executable_dictionary/test.py::test_executable_source_python [gw3] [ 26%] PASSED test_format_avro_confluent/test.py::test_select test_format_avro_confluent/test.py::test_select_auth [gw1] [ 27%] PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_range-sipHash64(key)] test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_sampling-key] [gw3] [ 28%] PASSED test_format_avro_confluent/test.py::test_select_auth test_format_avro_confluent/test.py::test_select_auth_encoded [gw0] [ 29%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-False-True] [gw2] [ 30%] PASSED test_executable_dictionary/test.py::test_executable_source_python test_executable_dictionary/test.py::test_executable_source_updated_python [gw3] [ 31%] PASSED test_format_avro_confluent/test.py::test_select_auth_encoded test_format_avro_confluent/test.py::test_select_auth_encoded_complex [gw3] [ 32%] PASSED test_format_avro_confluent/test.py::test_select_auth_encoded_complex [gw6] [ 33%] FAILED test_keeper_reconfig_remove_many/test.py::test_reconfig_remove_2_and_leader test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-True-False] [gw8] [ 34%] PASSED test_groupBitmapAnd_on_distributed/test_groupBitmapAndState_on_distributed_table.py::test_groupBitmapAndState_on_distributed_table [gw1] [ 35%] FAILED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_sampling-key] test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)] [gw1] [ 36%] PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)] test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_range-key] [gw7] [ 37%] PASSED test_log_query_probability/test.py::test_log_quries_probability_one test_log_query_probability/test.py::test_log_quries_probability_two [gw9] [ 38%] PASSED test_keeper_profiler/test.py::test_profiler [gw4] [ 39%] PASSED test_polymorphic_parts/test.py::test_compact_parts_only test_polymorphic_parts/test.py::test_different_part_types_on_replicas[polymorphic_table_compact-Compact] [gw0] [ 40%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-True-False] [gw5] [ 41%] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-10000-SELECT sum(key) FROM {table_name}] test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-100000-SELECT sum(key) FROM {table_name}] test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-True-True] [gw1] [ 42%] PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_range-key] test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_range-sipHash64(key)] test_keeper_snapshot_small_distance/test.py::test_snapshot_and_load [gw4] [ 43%] PASSED test_polymorphic_parts/test.py::test_different_part_types_on_replicas[polymorphic_table_compact-Compact] test_polymorphic_parts/test.py::test_different_part_types_on_replicas[polymorphic_table_wide-Wide] test_log_levels_update/test.py::test_log_levels_update [gw2] [ 44%] PASSED test_executable_dictionary/test.py::test_executable_source_updated_python [gw1] [ 45%] PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_range-sipHash64(key)] test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_sampling-key] [gw4] [ 46%] PASSED test_polymorphic_parts/test.py::test_different_part_types_on_replicas[polymorphic_table_wide-Wide] test_polymorphic_parts/test.py::test_polymorphic_parts_basics[first_node0-second_node0] test_profile_events_s3/test.py::test_profile_events [gw0] [ 47%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-True-True] test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-False-False] [gw1] [ 48%] PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_sampling-key] test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)] [gw1] [ 49%] PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)] [gw0] [ 50%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-False-False] [gw5] [ 51%] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-100000-SELECT sum(key) FROM {table_name}] test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-1000-SELECT sum(key) FROM {table_name}] test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-False-True] test_keeper_restore_from_snapshot/test_disk_s3.py::test_recover_from_snapshot_with_disk_s3 [gw6] [ 52%] PASSED test_log_levels_update/test.py::test_log_levels_update [gw0] [ 53%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-False-True] test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-True-False] test_merge_tree_check_part_with_cache/test.py::test_check_part_with_cache [gw5] [ 54%] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-1000-SELECT sum(key) FROM {table_name}] test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-10000-SELECT sum(key) FROM {table_name}] [gw0] [ 55%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-True-False] [gw2] [ 56%] PASSED test_profile_events_s3/test.py::test_profile_events test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-True-True] [gw6] [ 57%] SKIPPED test_merge_tree_check_part_with_cache/test.py::test_check_part_with_cache [gw0] [ 58%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-True-True] test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-False-False] [gw5] [ 59%] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-10000-SELECT sum(key) FROM {table_name}] test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-100000-SELECT sum(key) FROM {table_name}] [gw4] [ 60%] PASSED test_polymorphic_parts/test.py::test_polymorphic_parts_basics[first_node0-second_node0] test_polymorphic_parts/test.py::test_polymorphic_parts_basics[first_node1-second_node1] [gw0] [ 61%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-False-False] test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-False-True] [gw9] [ 62%] PASSED test_keeper_restore_from_snapshot/test_disk_s3.py::test_recover_from_snapshot_with_disk_s3 [gw0] [ 63%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-False-True] test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-True-False] [gw5] [ 64%] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-100000-SELECT sum(key) FROM {table_name}] test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-1000-SELECT sum(key) FROM {table_name}] [gw0] [ 65%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-True-False] test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-True-True] [gw0] [ 66%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-True-True] [gw5] [ 67%] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-1000-SELECT sum(key) FROM {table_name}] test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-10000-SELECT sum(key) FROM {table_name}] test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-False-False] test_parallel_replicas_increase_error_count/test.py::test_increase_error_count [gw8] [ 68%] PASSED test_keeper_snapshot_small_distance/test.py::test_snapshot_and_load [gw0] [ 69%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-False-False] test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-False-True] test_match_process_uid_against_data_owner/test.py::test_different_user [gw5] [ 70%] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-10000-SELECT sum(key) FROM {table_name}] test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-100000-SELECT sum(key) FROM {table_name}] [gw4] [ 71%] PASSED test_polymorphic_parts/test.py::test_polymorphic_parts_basics[first_node1-second_node1] test_polymorphic_parts/test.py::test_polymorphic_parts_index [gw0] [ 72%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-False-True] test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-True-False] [gw4] [ 73%] PASSED test_polymorphic_parts/test.py::test_polymorphic_parts_index test_polymorphic_parts/test.py::test_polymorphic_parts_non_adaptive [gw4] [ 74%] PASSED test_polymorphic_parts/test.py::test_polymorphic_parts_non_adaptive [gw0] [ 75%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-True-False] test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-True-True] [gw5] [ 76%] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-100000-SELECT sum(key) FROM {table_name}] test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-1000-SELECT sum(key) FROM {table_name}] [gw8] [ 77%] PASSED test_match_process_uid_against_data_owner/test.py::test_different_user [gw0] [ 78%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-True-True] test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-False-False] [gw0] [ 79%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-False-False] test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-False-True] [gw9] [ 80%] PASSED test_parallel_replicas_increase_error_count/test.py::test_increase_error_count [gw5] [ 81%] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-1000-SELECT sum(key) FROM {table_name}] test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-10000-SELECT sum(key) FROM {table_name}] [gw0] [ 82%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-False-True] test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-True-False] [gw5] [ 83%] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-10000-SELECT sum(key) FROM {table_name}] test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-100000-SELECT sum(key) FROM {table_name}] [gw0] [ 84%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-True-False] test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-True-True] [gw0] [ 85%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-True-True] test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-False-False] [gw5] [ 86%] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-100000-SELECT sum(key) FROM {table_name}] [gw0] [ 87%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-False-False] test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-False-True] [gw0] [ 88%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-False-True] test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-True-False] [gw0] [ 89%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-True-False] test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-True-True] [gw0] [ 90%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-True-True] test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-False-False] [gw0] [ 91%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-False-False] test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-False-True] [gw0] [ 92%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-False-True] test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-True-False] [gw0] [ 93%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-True-False] test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-True-True] [gw0] [ 94%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-True-True] test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-False-False] [gw0] [ 95%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-False-False] test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-False-True] [gw0] [ 96%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-False-True] test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-True-False] [gw0] [ 97%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-True-False] test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-True-True] [gw0] [ 98%] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-True-True] [gw7] [ 99%] PASSED test_log_query_probability/test.py::test_log_quries_probability_two test_grant_and_revoke/test_without_table_engine_grant.py::test_table_engine_and_source_grant [gw7] [100%] PASSED test_grant_and_revoke/test_without_table_engine_grant.py::test_table_engine_and_source_grant =================================== FAILURES =================================== ______________________ test_reconfig_remove_2_and_leader _______________________ [gw6] linux -- Python 3.10.12 /usr/bin/python3 started_cluster = def test_reconfig_remove_2_and_leader(started_cluster): """ Remove 2 followers from a cluster of 5. Remove leader from 3 nodes. """ global zk1, zk2, zk3, zk4, zk5 zk1 = create_client(node1) config = ku.get_config_str(zk1) print("Initial config", config) assert len(config.split("\n")) == 5 for i in range(100): zk1.create(f"/test_two_{i}", "somedata") zk4 = create_client(node4) zk4.sync("/test_two_0") ku.wait_configs_equal(config, zk4) zk5 = create_client(node5) zk5.sync("/test_two_0") ku.wait_configs_equal(config, zk5) for i in range(100): assert zk4.exists(f"test_two_{i}") assert zk5.exists(f"test_two_{i}") zk4.create(f"/test_two_{100 + i}", "otherdata") > zk2 = create_client(node2) test_keeper_reconfig_remove_many/test.py:73: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ test_keeper_reconfig_remove_many/test.py:39: in create_client return ku.KeeperClient( helpers/keeper_utils.py:118: in __init__ self.get("/keeper", 60.0) helpers/keeper_utils.py:178: in get return self.execute_query(f"get '{path}'", timeout) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = query = "get '/keeper'", timeout = 60.0 def execute_query(self, query: str, timeout: float = 60.0) -> str: output = io.BytesIO() self.proc.stdin.write(query.encode() + b"\n") self.proc.stdin.flush() events = self.poller.poll(timeout) if not events: raise TimeoutError(f"Keeper client returned no output") for fd_num, event in events: if event & (select.EPOLLIN | select.EPOLLPRI): file = self._fd_nums[fd_num] if file == self.proc.stdout: while True: chunk = file.readline() if chunk.endswith(self.SEPARATOR): break output.write(chunk) elif file == self.proc.stderr: self.proc.stdout.readline() > raise KeeperException(self.proc.stderr.readline().strip().decode()) E helpers.keeper_utils.KeeperException: Failed to connect to ZooKeeper: Code: 999. Coordination::Exception: Failed to get feature flags. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): helpers/keeper_utils.py:160: KeeperException ---------------------------- Captured stdout setup ----------------------------- Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml ------------------------------ Captured log setup ------------------------------ 2026-04-10 18:23:07.414000 [ 657 ] INFO : Running tests in /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/test.py (cluster.py:2748, start) 2026-04-10 18:23:07.414000 [ 657 ] DEBUG : Cluster start called. is_up=False (cluster.py:2755, start) 2026-04-10 18:23:07.459000 [ 657 ] DEBUG : Docker networks for project roottestkeeperreconfigremovemany-gw6 are NETWORK ID NAME DRIVER SCOPE (cluster.py:840, print_all_docker_pieces) 2026-04-10 18:23:07.489000 [ 657 ] DEBUG : Docker containers for project roottestkeeperreconfigremovemany-gw6 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:848, print_all_docker_pieces) 2026-04-10 18:23:07.513000 [ 657 ] DEBUG : Docker volumes for project roottestkeeperreconfigremovemany-gw6 are DRIVER VOLUME NAME (cluster.py:856, print_all_docker_pieces) 2026-04-10 18:23:07.513000 [ 657 ] DEBUG : Cleanup called (cluster.py:861, cleanup) 2026-04-10 18:23:07.542000 [ 657 ] DEBUG : Docker networks for project roottestkeeperreconfigremovemany-gw6 are NETWORK ID NAME DRIVER SCOPE (cluster.py:840, print_all_docker_pieces) 2026-04-10 18:23:07.564000 [ 657 ] DEBUG : Docker containers for project roottestkeeperreconfigremovemany-gw6 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:848, print_all_docker_pieces) 2026-04-10 18:23:07.591000 [ 657 ] DEBUG : Docker volumes for project roottestkeeperreconfigremovemany-gw6 are DRIVER VOLUME NAME (cluster.py:856, print_all_docker_pieces) 2026-04-10 18:23:07.592000 [ 657 ] DEBUG : Command:[docker container list --all --filter name='^/roottestkeeperreconfigremovemany-gw6-.*-1$' --format '{{.ID}}:{{.Names}}'] (cluster.py:121, run_and_check) 2026-04-10 18:23:07.646000 [ 657 ] DEBUG : Unstopped containers: {} (cluster.py:875, cleanup) 2026-04-10 18:23:07.646000 [ 657 ] DEBUG : No running containers for project: roottestkeeperreconfigremovemany-gw6 (cluster.py:889, cleanup) 2026-04-10 18:23:07.646000 [ 657 ] DEBUG : Trying to prune unused networks... (cluster.py:895, cleanup) 2026-04-10 18:23:07.687000 [ 657 ] DEBUG : Trying to prune unused images... (cluster.py:911, cleanup) 2026-04-10 18:23:07.687000 [ 657 ] DEBUG : Command:[docker image prune -f] (cluster.py:121, run_and_check) 2026-04-10 18:23:07.728000 [ 657 ] DEBUG : Stdout:Total reclaimed space: 0B (cluster.py:145, run_and_check) 2026-04-10 18:23:07.729000 [ 657 ] DEBUG : Images pruned (cluster.py:914, cleanup) 2026-04-10 18:23:07.729000 [ 657 ] DEBUG : Trying to prune unused volumes... (cluster.py:920, cleanup) 2026-04-10 18:23:07.729000 [ 657 ] DEBUG : Command:[docker volume ls | wc -l] (cluster.py:121, run_and_check) 2026-04-10 18:23:07.755000 [ 657 ] DEBUG : Stdout:6 (cluster.py:145, run_and_check) 2026-04-10 18:23:07.755000 [ 657 ] DEBUG : Command:[docker volume prune -f] (cluster.py:121, run_and_check) 2026-04-10 18:23:07.786000 [ 657 ] DEBUG : Stdout:Total reclaimed space: 0B (cluster.py:145, run_and_check) 2026-04-10 18:23:07.787000 [ 657 ] DEBUG : Volumes pruned: 6 (cluster.py:925, cleanup) 2026-04-10 18:23:07.787000 [ 657 ] DEBUG : Setup directory for instance: node1 (cluster.py:2768, start) 2026-04-10 18:23:07.788000 [ 657 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4638, create_dir) 2026-04-10 18:23:07.788000 [ 657 ] DEBUG : Create directory for common tests configuration (cluster.py:4643, create_dir) 2026-04-10 18:23:07.788000 [ 657 ] DEBUG : Copy common configuration from helpers (cluster.py:4663, create_dir) 2026-04-10 18:23:07.789000 [ 657 ] DEBUG : Generate and write macros file (cluster.py:4715, create_dir) 2026-04-10 18:23:07.789000 [ 657 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_keeper_reconfig_remove_many/configs/keeper1.xml'] to /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node1/configs/config.d (cluster.py:4755, create_dir) 2026-04-10 18:23:07.790000 [ 657 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node1/database (cluster.py:4772, create_dir) 2026-04-10 18:23:07.791000 [ 657 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node1/logs (cluster.py:4783, create_dir) 2026-04-10 18:23:07.791000 [ 657 ] DEBUG : Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] (cluster.py:4864, create_dir) 2026-04-10 18:23:07.791000 [ 657 ] DEBUG : Setup directory for instance: node2 (cluster.py:2768, start) 2026-04-10 18:23:07.792000 [ 657 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4638, create_dir) 2026-04-10 18:23:07.792000 [ 657 ] DEBUG : Create directory for common tests configuration (cluster.py:4643, create_dir) 2026-04-10 18:23:07.792000 [ 657 ] DEBUG : Copy common configuration from helpers (cluster.py:4663, create_dir) 2026-04-10 18:23:07.793000 [ 657 ] DEBUG : Generate and write macros file (cluster.py:4715, create_dir) 2026-04-10 18:23:07.793000 [ 657 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_keeper_reconfig_remove_many/configs/keeper2.xml'] to /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node2/configs/config.d (cluster.py:4755, create_dir) 2026-04-10 18:23:07.794000 [ 657 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node2/database (cluster.py:4772, create_dir) 2026-04-10 18:23:07.794000 [ 657 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node2/logs (cluster.py:4783, create_dir) 2026-04-10 18:23:07.795000 [ 657 ] DEBUG : Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] (cluster.py:4864, create_dir) 2026-04-10 18:23:07.795000 [ 657 ] DEBUG : Setup directory for instance: node3 (cluster.py:2768, start) 2026-04-10 18:23:07.795000 [ 657 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4638, create_dir) 2026-04-10 18:23:07.795000 [ 657 ] DEBUG : Create directory for common tests configuration (cluster.py:4643, create_dir) 2026-04-10 18:23:07.796000 [ 657 ] DEBUG : Copy common configuration from helpers (cluster.py:4663, create_dir) 2026-04-10 18:23:07.796000 [ 657 ] DEBUG : Generate and write macros file (cluster.py:4715, create_dir) 2026-04-10 18:23:07.797000 [ 657 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_keeper_reconfig_remove_many/configs/keeper3.xml'] to /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node3/configs/config.d (cluster.py:4755, create_dir) 2026-04-10 18:23:07.797000 [ 657 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node3/database (cluster.py:4772, create_dir) 2026-04-10 18:23:07.797000 [ 657 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node3/logs (cluster.py:4783, create_dir) 2026-04-10 18:23:07.798000 [ 657 ] DEBUG : Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] (cluster.py:4864, create_dir) 2026-04-10 18:23:07.798000 [ 657 ] DEBUG : Setup directory for instance: node4 (cluster.py:2768, start) 2026-04-10 18:23:07.798000 [ 657 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4638, create_dir) 2026-04-10 18:23:07.799000 [ 657 ] DEBUG : Create directory for common tests configuration (cluster.py:4643, create_dir) 2026-04-10 18:23:07.799000 [ 657 ] DEBUG : Copy common configuration from helpers (cluster.py:4663, create_dir) 2026-04-10 18:23:07.799000 [ 657 ] DEBUG : Generate and write macros file (cluster.py:4715, create_dir) 2026-04-10 18:23:07.800000 [ 657 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_keeper_reconfig_remove_many/configs/keeper4.xml'] to /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node4/configs/config.d (cluster.py:4755, create_dir) 2026-04-10 18:23:07.800000 [ 657 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node4/database (cluster.py:4772, create_dir) 2026-04-10 18:23:07.801000 [ 657 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node4/logs (cluster.py:4783, create_dir) 2026-04-10 18:23:07.801000 [ 657 ] DEBUG : Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] (cluster.py:4864, create_dir) 2026-04-10 18:23:07.801000 [ 657 ] DEBUG : Setup directory for instance: node5 (cluster.py:2768, start) 2026-04-10 18:23:07.802000 [ 657 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4638, create_dir) 2026-04-10 18:23:07.802000 [ 657 ] DEBUG : Create directory for common tests configuration (cluster.py:4643, create_dir) 2026-04-10 18:23:07.802000 [ 657 ] DEBUG : Copy common configuration from helpers (cluster.py:4663, create_dir) 2026-04-10 18:23:07.803000 [ 657 ] DEBUG : Generate and write macros file (cluster.py:4715, create_dir) 2026-04-10 18:23:07.803000 [ 657 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_keeper_reconfig_remove_many/configs/keeper5.xml'] to /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node5/configs/config.d (cluster.py:4755, create_dir) 2026-04-10 18:23:07.804000 [ 657 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node5/database (cluster.py:4772, create_dir) 2026-04-10 18:23:07.804000 [ 657 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node5/logs (cluster.py:4783, create_dir) 2026-04-10 18:23:07.804000 [ 657 ] DEBUG : Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] (cluster.py:4864, create_dir) 2026-04-10 18:23:07.804000 [ 657 ] DEBUG : Env {'ASAN_OPTIONS': 'use_sigaltstack=0', 'TSAN_OPTIONS': 'use_sigaltstack=0', 'LSAN_OPTIONS': 'suppressions=/etc/clickhouse-server/lsan_suppressions.txt', 'CLICKHOUSE_WATCHDOG_ENABLE': '0', 'CLICKHOUSE_NATS_TLS_SECURE': '0', 'LLVM_PROFILE_FILE': '/var/lib/clickhouse/server_%h_%p_%m.profraw'} stored in /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/.env (cluster.py:96, _create_env_file) 2026-04-10 18:23:07.805000 [ 657 ] DEBUG : Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] (config.py:21, find_config_file) 2026-04-10 18:23:07.805000 [ 657 ] DEBUG : No config file found (config.py:28, find_config_file) 2026-04-10 18:23:07.805000 [ 657 ] DEBUG : Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] (config.py:21, find_config_file) 2026-04-10 18:23:07.805000 [ 657 ] DEBUG : No config file found (config.py:28, find_config_file) 2026-04-10 18:23:07.825000 [ 657 ] DEBUG : http://localhost:None "GET /version HTTP/1.1" 200 826 (connectionpool.py:547, _make_request) 2026-04-10 18:23:07.826000 [ 657 ] DEBUG : Command:[docker compose --env-file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/.env --project-name roottestkeeperreconfigremovemany-gw6 --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node1/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node2/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node3/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node4/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node5/docker-compose.yml pull] (cluster.py:121, run_and_check) 2026-04-10 18:23:16.645000 [ 657 ] DEBUG : Stderr: node1 Skipped - Image is already being pulled by node5 (cluster.py:147, run_and_check) 2026-04-10 18:23:16.651000 [ 657 ] DEBUG : Stderr: node2 Skipped - Image is already being pulled by node5 (cluster.py:147, run_and_check) 2026-04-10 18:23:16.651000 [ 657 ] DEBUG : Stderr: node3 Skipped - Image is already being pulled by node5 (cluster.py:147, run_and_check) 2026-04-10 18:23:16.651000 [ 657 ] DEBUG : Stderr: node4 Skipped - Image is already being pulled by node5 (cluster.py:147, run_and_check) 2026-04-10 18:23:16.651000 [ 657 ] DEBUG : Stderr: node5 Pulling (cluster.py:147, run_and_check) 2026-04-10 18:23:16.651000 [ 657 ] DEBUG : Stderr: node5 Pulled (cluster.py:147, run_and_check) 2026-04-10 18:23:16.652000 [ 657 ] DEBUG : ('Trying to create ClickHouse instance by command %s', 'docker compose --env-file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/.env --project-name roottestkeeperreconfigremovemany-gw6 --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node1/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node2/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node3/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node4/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node5/docker-compose.yml up -d --no-recreate') (cluster.py:3149, start) 2026-04-10 18:23:16.653000 [ 657 ] DEBUG : Command:[docker compose --env-file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/.env --project-name roottestkeeperreconfigremovemany-gw6 --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node1/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node2/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node3/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node4/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node5/docker-compose.yml up -d --no-recreate] (cluster.py:121, run_and_check) 2026-04-10 18:23:19.169000 [ 657 ] DEBUG : Stderr: Network roottestkeeperreconfigremovemany-gw6_default Creating (cluster.py:147, run_and_check) 2026-04-10 18:23:19.170000 [ 657 ] DEBUG : Stderr: Network roottestkeeperreconfigremovemany-gw6_default Created (cluster.py:147, run_and_check) 2026-04-10 18:23:19.170000 [ 657 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node4-1 Creating (cluster.py:147, run_and_check) 2026-04-10 18:23:19.170000 [ 657 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node5-1 Creating (cluster.py:147, run_and_check) 2026-04-10 18:23:19.170000 [ 657 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node1-1 Creating (cluster.py:147, run_and_check) 2026-04-10 18:23:19.170000 [ 657 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node2-1 Creating (cluster.py:147, run_and_check) 2026-04-10 18:23:19.170000 [ 657 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node3-1 Creating (cluster.py:147, run_and_check) 2026-04-10 18:23:19.171000 [ 657 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node2-1 Created (cluster.py:147, run_and_check) 2026-04-10 18:23:19.171000 [ 657 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node1-1 Created (cluster.py:147, run_and_check) 2026-04-10 18:23:19.171000 [ 657 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node5-1 Created (cluster.py:147, run_and_check) 2026-04-10 18:23:19.171000 [ 657 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node4-1 Created (cluster.py:147, run_and_check) 2026-04-10 18:23:19.171000 [ 657 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node3-1 Created (cluster.py:147, run_and_check) 2026-04-10 18:23:19.171000 [ 657 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node5-1 Starting (cluster.py:147, run_and_check) 2026-04-10 18:23:19.171000 [ 657 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node1-1 Starting (cluster.py:147, run_and_check) 2026-04-10 18:23:19.171000 [ 657 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node2-1 Starting (cluster.py:147, run_and_check) 2026-04-10 18:23:19.172000 [ 657 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node3-1 Starting (cluster.py:147, run_and_check) 2026-04-10 18:23:19.172000 [ 657 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node4-1 Starting (cluster.py:147, run_and_check) 2026-04-10 18:23:19.172000 [ 657 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node1-1 Started (cluster.py:147, run_and_check) 2026-04-10 18:23:19.172000 [ 657 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node2-1 Started (cluster.py:147, run_and_check) 2026-04-10 18:23:19.172000 [ 657 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node3-1 Started (cluster.py:147, run_and_check) 2026-04-10 18:23:19.172000 [ 657 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node5-1 Started (cluster.py:147, run_and_check) 2026-04-10 18:23:19.172000 [ 657 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node4-1 Started (cluster.py:147, run_and_check) 2026-04-10 18:23:19.172000 [ 657 ] DEBUG : ClickHouse instance created (cluster.py:3157, start) 2026-04-10 18:23:19.173000 [ 657 ] DEBUG : get_instance_ip instance_name=node1 (cluster.py:2015, get_instance_ip) 2026-04-10 18:23:19.209000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw6-node1-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:19.214000 [ 657 ] DEBUG : get_instance_ip instance_name=node1 (cluster.py:2025, get_instance_global_ipv6) 2026-04-10 18:23:19.218000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw6-node1-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:19.219000 [ 657 ] DEBUG : Waiting for ClickHouse start in node1, ip: 172.16.6.2... (cluster.py:3165, start) 2026-04-10 18:23:19.222000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw6-node1-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:19.224000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:19.329000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:19.442000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:19.555000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:19.665000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:19.771000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:19.875000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:19.983000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:20.088000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:20.193000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:20.305000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:20.414000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:20.519000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:20.634000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:20.741000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:20.850000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:20.955000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:21.061000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:21.164000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:21.269000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:21.381000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:21.486000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:21.591000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:21.695000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:21.800000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:21.904000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:22.009000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:22.117000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:22.224000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:22.332000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:22.436000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:22.543000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:22.648000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:22.752000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:22.857000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:22.965000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:23.072000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:23.181000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:23.287000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:23.404000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:23.512000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:23.616000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:23.736000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:23.843000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:23.949000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:24.067000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:24.172000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:24.276000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:24.386000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:24.493000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:24.601000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:24.717000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:24.826000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:24.931000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:25.037000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:25.142000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:25.248000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:25.353000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:25.558000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:25.665000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:25.771000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:25.882000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:25.987000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:26.090000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:26.206000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:26.311000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:26.463000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:26.587000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:26.698000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:26.807000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:26.916000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:27.022000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:27.126000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:27.231000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:27.335000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:27.439000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:27.543000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:27.650000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:27.758000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:27.876000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:27.982000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/d039fca9f879b2f89902de9a71ca3ba0931b318684019e97a103c0664111d343/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:27.984000 [ 657 ] DEBUG : ClickHouse node1 started (cluster.py:3169, start) 2026-04-10 18:23:27.984000 [ 657 ] DEBUG : get_instance_ip instance_name=node2 (cluster.py:2015, get_instance_ip) 2026-04-10 18:23:27.995000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw6-node2-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:27.996000 [ 657 ] DEBUG : get_instance_ip instance_name=node2 (cluster.py:2025, get_instance_global_ipv6) 2026-04-10 18:23:27.998000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw6-node2-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:27.999000 [ 657 ] DEBUG : Waiting for ClickHouse start in node2, ip: 172.16.6.6... (cluster.py:3165, start) 2026-04-10 18:23:28.014000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw6-node2-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:28.017000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/98804941b6a2443e00fd304055d4d573cf5c562c02cac430dbf892e088d4448a/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:28.018000 [ 657 ] DEBUG : ClickHouse node2 started (cluster.py:3169, start) 2026-04-10 18:23:28.018000 [ 657 ] DEBUG : get_instance_ip instance_name=node3 (cluster.py:2015, get_instance_ip) 2026-04-10 18:23:28.024000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw6-node3-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:28.026000 [ 657 ] DEBUG : get_instance_ip instance_name=node3 (cluster.py:2025, get_instance_global_ipv6) 2026-04-10 18:23:28.034000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw6-node3-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:28.035000 [ 657 ] DEBUG : Waiting for ClickHouse start in node3, ip: 172.16.6.4... (cluster.py:3165, start) 2026-04-10 18:23:28.037000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw6-node3-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:28.045000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/66292d1b8fbe9a08c2eeffcd45b145af33debb378eb156ea939f124b285d304c/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:28.046000 [ 657 ] DEBUG : ClickHouse node3 started (cluster.py:3169, start) 2026-04-10 18:23:28.046000 [ 657 ] DEBUG : get_instance_ip instance_name=node4 (cluster.py:2015, get_instance_ip) 2026-04-10 18:23:28.056000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw6-node4-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:28.057000 [ 657 ] DEBUG : get_instance_ip instance_name=node4 (cluster.py:2025, get_instance_global_ipv6) 2026-04-10 18:23:28.064000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw6-node4-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:28.065000 [ 657 ] DEBUG : Waiting for ClickHouse start in node4, ip: 172.16.6.5... (cluster.py:3165, start) 2026-04-10 18:23:28.067000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw6-node4-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:28.073000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/2b8cdb4b367e2cb317f9c12a7faec377e07acb677e04dbb88f9aca71046fa1a1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:28.075000 [ 657 ] DEBUG : ClickHouse node4 started (cluster.py:3169, start) 2026-04-10 18:23:28.076000 [ 657 ] DEBUG : get_instance_ip instance_name=node5 (cluster.py:2015, get_instance_ip) 2026-04-10 18:23:28.079000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw6-node5-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:28.080000 [ 657 ] DEBUG : get_instance_ip instance_name=node5 (cluster.py:2025, get_instance_global_ipv6) 2026-04-10 18:23:28.083000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw6-node5-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:28.084000 [ 657 ] DEBUG : Waiting for ClickHouse start in node5, ip: 172.16.6.3... (cluster.py:3165, start) 2026-04-10 18:23:28.086000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw6-node5-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:28.088000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/93afdd142211c88f32a9313d9f1ef92539f8ee23928a446bbdfa8fbfe851f64a/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:28.089000 [ 657 ] DEBUG : ClickHouse node5 started (cluster.py:3169, start) ----------------------------- Captured stdout call ----------------------------- Initial config server.1=node1:9234;participant;1 server.2=node2:9234;participant;1 server.3=node3:9234;participant;1 server.4=node4:9234;participant;1 server.5=node5:9234;participant;1 ------------------------------ Captured log call ------------------------------- 2026-04-10 18:23:28.125000 [ 657 ] DEBUG : get_instance_ip instance_name=node1 (cluster.py:2015, get_instance_ip) 2026-04-10 18:23:28.128000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw6-node1-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:30.709000 [ 657 ] DEBUG : get_instance_ip instance_name=node4 (cluster.py:2015, get_instance_ip) 2026-04-10 18:23:30.715000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw6-node4-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:31.127000 [ 657 ] DEBUG : get_instance_ip instance_name=node5 (cluster.py:2015, get_instance_ip) 2026-04-10 18:23:31.129000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw6-node5-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-04-10 18:23:41.730000 [ 657 ] DEBUG : get_instance_ip instance_name=node2 (cluster.py:2015, get_instance_ip) 2026-04-10 18:23:41.751000 [ 657 ] DEBUG : http://localhost:None "GET /v1.46/containers/roottestkeeperreconfigremovemany-gw6-node2-1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) ---------------------------- Captured log teardown ----------------------------- 2026-04-10 18:23:50.463000 [ 657 ] DEBUG : Command:[docker compose --env-file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/.env --project-name roottestkeeperreconfigremovemany-gw6 --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node1/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node2/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node3/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node4/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node5/docker-compose.yml stop --timeout 20] (cluster.py:121, run_and_check) 2026-04-10 18:24:12.593000 [ 657 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node4-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:24:12.600000 [ 657 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node1-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:24:12.600000 [ 657 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node2-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:24:12.602000 [ 657 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node3-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:24:12.602000 [ 657 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node5-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:24:12.602000 [ 657 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node1-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:24:12.602000 [ 657 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node2-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:24:12.602000 [ 657 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node4-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:24:12.602000 [ 657 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node3-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:24:12.603000 [ 657 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node5-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:24:12.604000 [ 657 ] DEBUG : Command:[bash -c [ -f /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node1/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node1/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] (cluster.py:121, run_and_check) 2026-04-10 18:24:12.672000 [ 657 ] DEBUG : Command:[bash -c [ -f /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node2/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node2/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] (cluster.py:121, run_and_check) 2026-04-10 18:24:12.728000 [ 657 ] DEBUG : Command:[bash -c [ -f /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node3/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node3/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] (cluster.py:121, run_and_check) 2026-04-10 18:24:12.741000 [ 657 ] DEBUG : Command:[bash -c [ -f /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node4/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node4/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] (cluster.py:121, run_and_check) 2026-04-10 18:24:12.769000 [ 657 ] DEBUG : Command:[bash -c [ -f /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node5/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node5/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] (cluster.py:121, run_and_check) 2026-04-10 18:24:12.809000 [ 657 ] DEBUG : Command:[docker compose --env-file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/.env --project-name roottestkeeperreconfigremovemany-gw6 --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node1/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node2/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node3/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node4/docker-compose.yml --file /ClickHouse/tests/integration/test_keeper_reconfig_remove_many/_instances-0-gw6/node5/docker-compose.yml down --volumes] (cluster.py:121, run_and_check) 2026-04-10 18:24:14.604000 [ 657 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node5-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:24:14.605000 [ 657 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node1-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:24:14.605000 [ 657 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node4-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:24:14.605000 [ 657 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node2-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:24:14.606000 [ 657 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node3-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 18:24:14.606000 [ 657 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node2-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:24:14.606000 [ 657 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node2-1 Removing (cluster.py:147, run_and_check) 2026-04-10 18:24:14.606000 [ 657 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node3-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:24:14.606000 [ 657 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node3-1 Removing (cluster.py:147, run_and_check) 2026-04-10 18:24:14.606000 [ 657 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node4-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:24:14.606000 [ 657 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node4-1 Removing (cluster.py:147, run_and_check) 2026-04-10 18:24:14.607000 [ 657 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node1-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:24:14.607000 [ 657 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node1-1 Removing (cluster.py:147, run_and_check) 2026-04-10 18:24:14.607000 [ 657 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node5-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 18:24:14.607000 [ 657 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node5-1 Removing (cluster.py:147, run_and_check) 2026-04-10 18:24:14.607000 [ 657 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node3-1 Removed (cluster.py:147, run_and_check) 2026-04-10 18:24:14.608000 [ 657 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node4-1 Removed (cluster.py:147, run_and_check) 2026-04-10 18:24:14.608000 [ 657 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node1-1 Removed (cluster.py:147, run_and_check) 2026-04-10 18:24:14.608000 [ 657 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node5-1 Removed (cluster.py:147, run_and_check) 2026-04-10 18:24:14.608000 [ 657 ] DEBUG : Stderr: Container roottestkeeperreconfigremovemany-gw6-node2-1 Removed (cluster.py:147, run_and_check) 2026-04-10 18:24:14.608000 [ 657 ] DEBUG : Stderr: Network roottestkeeperreconfigremovemany-gw6_default Removing (cluster.py:147, run_and_check) 2026-04-10 18:24:14.608000 [ 657 ] DEBUG : Stderr: Network roottestkeeperreconfigremovemany-gw6_default Removed (cluster.py:147, run_and_check) 2026-04-10 18:24:14.610000 [ 657 ] DEBUG : Cleanup called (cluster.py:861, cleanup) 2026-04-10 18:24:14.673000 [ 657 ] DEBUG : Docker networks for project roottestkeeperreconfigremovemany-gw6 are NETWORK ID NAME DRIVER SCOPE (cluster.py:840, print_all_docker_pieces) 2026-04-10 18:24:14.761000 [ 657 ] DEBUG : Docker containers for project roottestkeeperreconfigremovemany-gw6 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:848, print_all_docker_pieces) 2026-04-10 18:24:14.829000 [ 657 ] DEBUG : Docker volumes for project roottestkeeperreconfigremovemany-gw6 are DRIVER VOLUME NAME (cluster.py:856, print_all_docker_pieces) 2026-04-10 18:24:14.830000 [ 657 ] DEBUG : Command:[docker container list --all --filter name='^/roottestkeeperreconfigremovemany-gw6-.*-1$' --format '{{.ID}}:{{.Names}}'] (cluster.py:121, run_and_check) 2026-04-10 18:24:14.896000 [ 657 ] DEBUG : Unstopped containers: {} (cluster.py:875, cleanup) 2026-04-10 18:24:14.897000 [ 657 ] DEBUG : No running containers for project: roottestkeeperreconfigremovemany-gw6 (cluster.py:889, cleanup) 2026-04-10 18:24:14.897000 [ 657 ] DEBUG : Trying to prune unused networks... (cluster.py:895, cleanup) 2026-04-10 18:24:14.941000 [ 657 ] DEBUG : Trying to prune unused images... (cluster.py:911, cleanup) 2026-04-10 18:24:14.942000 [ 657 ] DEBUG : Command:[docker image prune -f] (cluster.py:121, run_and_check) 2026-04-10 18:24:15.025000 [ 657 ] DEBUG : Stdout:Total reclaimed space: 0B (cluster.py:145, run_and_check) 2026-04-10 18:24:15.025000 [ 657 ] DEBUG : Images pruned (cluster.py:914, cleanup) 2026-04-10 18:24:15.025000 [ 657 ] DEBUG : Trying to prune unused volumes... (cluster.py:920, cleanup) 2026-04-10 18:24:15.025000 [ 657 ] DEBUG : Command:[docker volume ls | wc -l] (cluster.py:121, run_and_check) 2026-04-10 18:24:15.065000 [ 657 ] DEBUG : Stdout:8 (cluster.py:145, run_and_check) 2026-04-10 18:24:15.065000 [ 657 ] DEBUG : Command:[docker volume prune -f] (cluster.py:121, run_and_check) 2026-04-10 18:24:15.110000 [ 657 ] DEBUG : Stdout:Total reclaimed space: 0B (cluster.py:145, run_and_check) 2026-04-10 18:24:15.111000 [ 657 ] DEBUG : Volumes pruned: 8 (cluster.py:925, cleanup) _ test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_sampling-key] _ [gw1] linux -- Python 3.10.12 /usr/bin/python3 start_cluster = cluster = 'test_single_shard_multiple_replicas', custom_key = 'key' parallel_replicas_mode = 'custom_key_sampling' @pytest.mark.parametrize("custom_key", ["sipHash64(key)", "key"]) @pytest.mark.parametrize( "parallel_replicas_mode", ["custom_key_sampling", "custom_key_range"] ) @pytest.mark.parametrize( "cluster", ["test_single_shard_multiple_replicas"], ) def test_parallel_replicas_custom_key_mergetree( start_cluster, cluster, custom_key, parallel_replicas_mode ): for node in nodes: node.rotate_logs() row_num = 1000 n1 = nodes[0] > n1.query(f"DROP TABLE IF EXISTS test_table_for_mt ON CLUSTER {cluster} SYNC") test_parallel_replicas_custom_key/test.py:120: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3659: in query return self.client.query( helpers/client.py:39: in wrap return func(self, *args, **kwargs) helpers/client.py:79: in query ).get_answer() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = def get_answer(self): self.process.wait(timeout=DEFAULT_QUERY_TIMEOUT) self.stdout_file.seek(0) self.stderr_file.seek(0) stdout = self.stdout_file.read().decode("utf-8", errors="replace") stderr = self.stderr_file.read().decode("utf-8", errors="replace") if ( self.timer is not None and not self.process_finished_before_timeout and not self.ignore_error ): logging.debug(f"Timed out. Last stdout:{stdout}, stderr:{stderr}") raise QueryTimeoutExceedException("Client timed out!") if ( self.process.returncode != 0 or self.remove_trash_from_stderr(stderr) ) and not self.ignore_error: > raise QueryRuntimeException( "Client failed! Return code: {}, stderr: {}".format( self.process.returncode, stderr ), self.process.returncode, stderr, ) E helpers.client.QueryRuntimeException: Client failed! Return code: 231, stderr: Received exception from server (version 25.3.8): E Code: 999. DB::Exception: Received from 172.16.1.5:9000. Coordination::Exception. Coordination::Exception: Coordination error: Connection loss, path /clickhouse/task_queue/ddl/query-. Stack trace: E E 0. ./contrib/llvm-project/libcxx/include/__exception/exception.h:113: Poco::Exception::Exception(String const&, int) @ 0x0000000036f39bb1 E 1. ./build_docker/./src/Common/Exception.cpp:108: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001b1bca51 E 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000b7f426b E 3. Coordination::Exception::Exception(Coordination::Error, FormatStringHelperImpl::type, std::type_identity::type>, char const*&&, String const&) @ 0x000000001002ab64 E 4. Coordination::Exception::fromPath(Coordination::Error, String const&) @ 0x0000000010029d7f E 5. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:497: zkutil::ZooKeeper::createAncestors(String const&) @ 0x00000000306c7282 E 6. ./build_docker/./src/Interpreters/DDLWorker.cpp:1087: DB::DDLWorker::enqueueQueryAttempt(DB::DDLLogEntry&) @ 0x0000000028bbbcac E 7. ./build_docker/./src/Interpreters/DDLWorker.cpp:1073: DB::DDLWorker::enqueueQuery(DB::DDLLogEntry&, DB::ZooKeeperRetriesInfo const&) @ 0x0000000028bbb17c E 8. ./build_docker/./src/Interpreters/executeDDLQueryOnCluster.cpp:192: DB::executeDDLQueryOnCluster(std::shared_ptr const&, std::shared_ptr, DB::DDLQueryOnClusterParams const&) @ 0x0000000029ff642f E 9. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:84: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000297a2ef1 E 10. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:72: DB::InterpreterDropQuery::execute() @ 0x00000000297a256d E 11. ./build_docker/./src/Interpreters/executeQuery.cpp:1457: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*, std::shared_ptr&) @ 0x000000002a01c743 E 12. ./build_docker/./src/Interpreters/executeQuery.cpp:1624: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x000000002a014665 E 13. ./build_docker/./src/Server/TCPHandler.cpp:664: DB::TCPHandler::runImpl() @ 0x000000002f19ad76 E 14. ./build_docker/./src/Server/TCPHandler.cpp:2629: DB::TCPHandler::run() @ 0x000000002f1daa5a E 15. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:40: Poco::Net::TCPServerConnection::start() @ 0x0000000037114faf E 16. ./build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115: Poco::Net::TCPServerDispatcher::run() @ 0x0000000037115c77 E 17. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:205: Poco::PooledThread::run() @ 0x00000000370244cb E 18. ./base/poco/Foundation/src/Thread_POSIX.cpp:335: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003701e0c8 E 19. asan_thread_start(void*) @ 0x000000000b7a6e77 E 20. ? @ 0x00007fa84297cac3 E 21. ? @ 0x00007fa842a0e850 E . (KEEPER_EXCEPTION) E (query: DROP TABLE IF EXISTS test_table_for_mt ON CLUSTER test_single_shard_multiple_replicas SYNC) helpers/client.py:248: QueryRuntimeException ------------------------------ Captured log call ------------------------------- 2026-04-10 18:23:41.184000 [ 642 ] DEBUG : run container_id:roottestparallelreplicascustomkey-gw1-n1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse server' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:2061, exec_in_container) 2026-04-10 18:23:41.184000 [ 642 ] DEBUG : Command:[docker exec roottestparallelreplicascustomkey-gw1-n1-1 bash -c ps ax | grep 'clickhouse server' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] (cluster.py:121, run_and_check) 2026-04-10 18:23:41.592000 [ 642 ] DEBUG : Stdout:1 (cluster.py:145, run_and_check) 2026-04-10 18:23:41.593000 [ 642 ] DEBUG : run container_id:roottestparallelreplicascustomkey-gw1-n1-1 detach:False nothrow:False cmd: ['bash', '-c', 'kill -HUP 1'] (cluster.py:2061, exec_in_container) 2026-04-10 18:23:41.593000 [ 642 ] DEBUG : Command:[docker exec -u root roottestparallelreplicascustomkey-gw1-n1-1 bash -c kill -HUP 1] (cluster.py:121, run_and_check) 2026-04-10 18:23:41.976000 [ 642 ] DEBUG : run container_id:roottestparallelreplicascustomkey-gw1-n2-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse server' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:2061, exec_in_container) 2026-04-10 18:23:41.976000 [ 642 ] DEBUG : Command:[docker exec roottestparallelreplicascustomkey-gw1-n2-1 bash -c ps ax | grep 'clickhouse server' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] (cluster.py:121, run_and_check) 2026-04-10 18:23:46.343000 [ 642 ] DEBUG : Stdout:1 (cluster.py:145, run_and_check) 2026-04-10 18:23:46.344000 [ 642 ] DEBUG : run container_id:roottestparallelreplicascustomkey-gw1-n2-1 detach:False nothrow:False cmd: ['bash', '-c', 'kill -HUP 1'] (cluster.py:2061, exec_in_container) 2026-04-10 18:23:46.344000 [ 642 ] DEBUG : Command:[docker exec -u root roottestparallelreplicascustomkey-gw1-n2-1 bash -c kill -HUP 1] (cluster.py:121, run_and_check) 2026-04-10 18:23:46.662000 [ 642 ] DEBUG : run container_id:roottestparallelreplicascustomkey-gw1-n3-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse server' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:2061, exec_in_container) 2026-04-10 18:23:46.663000 [ 642 ] DEBUG : Command:[docker exec roottestparallelreplicascustomkey-gw1-n3-1 bash -c ps ax | grep 'clickhouse server' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] (cluster.py:121, run_and_check) 2026-04-10 18:23:46.992000 [ 642 ] DEBUG : Stdout:1 (cluster.py:145, run_and_check) 2026-04-10 18:23:46.993000 [ 642 ] DEBUG : run container_id:roottestparallelreplicascustomkey-gw1-n3-1 detach:False nothrow:False cmd: ['bash', '-c', 'kill -HUP 1'] (cluster.py:2061, exec_in_container) 2026-04-10 18:23:46.993000 [ 642 ] DEBUG : Command:[docker exec -u root roottestparallelreplicascustomkey-gw1-n3-1 bash -c kill -HUP 1] (cluster.py:121, run_and_check) 2026-04-10 18:23:47.542000 [ 642 ] DEBUG : run container_id:roottestparallelreplicascustomkey-gw1-n4-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse server' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] (cluster.py:2061, exec_in_container) 2026-04-10 18:23:47.543000 [ 642 ] DEBUG : Command:[docker exec roottestparallelreplicascustomkey-gw1-n4-1 bash -c ps ax | grep 'clickhouse server' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] (cluster.py:121, run_and_check) 2026-04-10 18:23:48.066000 [ 642 ] DEBUG : Stdout:1 (cluster.py:145, run_and_check) 2026-04-10 18:23:48.067000 [ 642 ] DEBUG : run container_id:roottestparallelreplicascustomkey-gw1-n4-1 detach:False nothrow:False cmd: ['bash', '-c', 'kill -HUP 1'] (cluster.py:2061, exec_in_container) 2026-04-10 18:23:48.067000 [ 642 ] DEBUG : Command:[docker exec -u root roottestparallelreplicascustomkey-gw1-n4-1 bash -c kill -HUP 1] (cluster.py:121, run_and_check) 2026-04-10 18:23:48.326000 [ 642 ] DEBUG : Executing query DROP TABLE IF EXISTS test_table_for_mt ON CLUSTER test_single_shard_multiple_replicas SYNC on n1 (cluster.py:3658, query) ----------------- generated report log file: parallel1_0.jsonl ----------------- ============================== slowest durations =============================== 371.24s call test_log_query_probability/test.py::test_log_quries_probability_two 73.90s call test_log_query_probability/test.py::test_log_quries_probability_one 61.88s call test_keeper_snapshot_small_distance/test.py::test_snapshot_and_load 59.06s call test_polymorphic_parts/test.py::test_compact_parts_only 56.87s setup test_format_avro_confluent/test.py::test_select 48.10s call test_polymorphic_parts/test.py::test_polymorphic_parts_basics[first_node0-second_node0] 43.52s call test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-10000-SELECT sum(key) FROM {table_name}] 38.94s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-False-False] 38.05s call test_polymorphic_parts/test.py::test_polymorphic_parts_basics[first_node1-second_node1] 35.81s setup test_keeper_profiler/test.py::test_profiler 34.01s setup test_groupBitmapAnd_on_distributed/test_groupBitmapAndState_on_distributed_table.py::test_groupBitmapAndState_on_distributed_table 32.03s setup test_polymorphic_parts/test.py::test_compact_parts_only 31.22s teardown test_format_avro_confluent/test.py::test_select_auth_encoded_complex 30.18s teardown test_kerberos_auth/test.py::test_kerberos_auth_without_keytab 28.69s call test_executable_dictionary/test.py::test_executable_source_updated_python 28.31s setup test_parallel_replicas_increase_error_count/test.py::test_increase_error_count 26.59s teardown test_keeper_reconfig_remove_many/test.py::test_reconfig_remove_2_and_leader 25.97s teardown test_keeper_profiler/test.py::test_profiler 25.81s setup test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-1000-SELECT sum(key) FROM {table_name}] 24.90s teardown test_keeper_restore_from_snapshot/test_disk_s3.py::test_recover_from_snapshot_with_disk_s3 24.09s setup test_executable_dictionary/test.py::test_executable_input_bash 23.32s teardown test_profile_events_s3/test.py::test_profile_events 22.84s call test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-1000-SELECT sum(key) FROM {table_name}] 22.52s setup test_keeper_restore_from_snapshot/test_disk_s3.py::test_recover_from_snapshot_with_disk_s3 22.40s call test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-100000-SELECT sum(key) FROM {table_name}] 22.05s teardown test_merge_tree_check_part_with_cache/test.py::test_check_part_with_cache 21.06s setup test_keeper_snapshot_small_distance/test.py::test_snapshot_and_load 20.77s setup test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_multiple_shards_multiple_replicas-custom_key_sampling-key] 20.70s setup test_keeper_reconfig_remove_many/test.py::test_reconfig_remove_2_and_leader 19.79s call test_keeper_reconfig_remove_many/test.py::test_reconfig_remove_2_and_leader 19.45s setup test_kerberos_auth/test.py::test_bad_path_to_keytab 18.31s teardown test_groupBitmapAnd_on_distributed/test_groupBitmapAndState_on_distributed_table.py::test_groupBitmapAndState_on_distributed_table 17.22s setup test_profile_events_s3/test.py::test_profile_events 16.72s setup test_password_constraints/test.py::test_complexity_rules 16.56s call test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-10000-SELECT sum(key) FROM {table_name}] 16.35s setup test_log_levels_update/test.py::test_log_levels_update 15.64s setup test_max_authentication_methods_per_user/test.py::test_alter 15.52s call test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-1000-SELECT sum(key) FROM {table_name}] 15.11s call test_groupBitmapAnd_on_distributed/test_groupBitmapAndState_on_distributed_table.py::test_groupBitmapAndState_on_distributed_table 14.71s call test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-False-True] 14.36s setup test_log_query_probability/test.py::test_log_quries_probability_one 13.81s call test_executable_dictionary/test.py::test_executable_source_python 13.76s call test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-100000-SELECT sum(key) FROM {table_name}] 13.60s call test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-10000-SELECT sum(key) FROM {table_name}] 13.43s teardown test_polymorphic_parts/test.py::test_polymorphic_parts_non_adaptive 13.34s call test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-100000-SELECT sum(key) FROM {table_name}] 13.31s call test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_sampling-key] 13.18s call test_keeper_restore_from_snapshot/test_disk_s3.py::test_recover_from_snapshot_with_disk_s3 13.10s setup test_grant_and_revoke/test_without_table_engine_grant.py::test_table_engine_and_source_grant 12.71s call test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-1000-SELECT sum(key) FROM {table_name}] 12.46s call test_profile_events_s3/test.py::test_profile_events 11.51s call test_executable_dictionary/test.py::test_executable_input_slow_python 11.14s call test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-1000-SELECT sum(key) FROM {table_name}] 11.09s teardown test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-100000-SELECT sum(key) FROM {table_name}] 11.09s teardown test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)] 11.09s call test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-10000-SELECT sum(key) FROM {table_name}] 11.00s call test_match_process_uid_against_data_owner/test.py::test_different_user 10.97s call test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_range-sipHash64(key)] 10.89s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-True-False] 10.77s call test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-100000-SELECT sum(key) FROM {table_name}] 10.26s setup test_merge_tree_check_part_with_cache/test.py::test_check_part_with_cache 8.96s call test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)] 8.52s call test_format_avro_confluent/test.py::test_select_auth 8.45s call test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-False-False] 8.30s call test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_range-key] 7.77s call test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)] 7.61s teardown test_parallel_replicas_increase_error_count/test.py::test_increase_error_count 7.61s teardown test_log_query_probability/test.py::test_log_quries_probability_two 7.48s setup test_match_process_uid_against_data_owner/test.py::test_different_user 7.34s call test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-False-False] 7.11s call test_executable_dictionary/test.py::test_executable_source_argument_python 7.10s call test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_range-key] 6.85s call test_parallel_replicas_increase_error_count/test.py::test_increase_error_count 6.35s call test_log_levels_update/test.py::test_log_levels_update 6.33s call test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-True-False] 6.27s call test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_multiple_shards_multiple_replicas-custom_key_sampling-key] 6.03s call test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_single_shard_multiple_replicas-custom_key_sampling-key] 6.01s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-False-True] 5.97s teardown test_max_authentication_methods_per_user/test.py::test_create_default_setting 5.78s call test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-True-True] 5.73s teardown test_password_constraints/test.py::test_default_password_type 5.46s call test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_range-sipHash64(key)] 5.35s call test_polymorphic_parts/test.py::test_different_part_types_on_replicas[polymorphic_table_compact-Compact] 5.26s call test_password_constraints/test.py::test_complexity_rules 5.09s call test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-True-False] 5.06s call test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-False-False] 5.05s call test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)] 4.96s call test_polymorphic_parts/test.py::test_different_part_types_on_replicas[polymorphic_table_wide-Wide] 4.95s call test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-True-False] 4.94s call test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-False-False] 4.80s call test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-True-True] 4.79s teardown test_grant_and_revoke/test_without_table_engine_grant.py::test_table_engine_and_source_grant 4.63s call test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_sampling-key] 4.41s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-False-True] 4.26s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-False-True] 4.17s call test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-True-True] 4.16s call test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_multiple_shards_multiple_replicas-custom_key_sampling-sipHash64(key)] 4.07s call test_format_avro_confluent/test.py::test_select 4.06s teardown test_log_levels_update/test.py::test_log_levels_update 4.02s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-True-True] 4.01s call test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-False-True] 3.88s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-True-True] 3.86s call test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-True-True] 3.79s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-False-True] 3.63s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-True-True] 3.63s call test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-False-True] 3.55s call test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-False-True] 3.52s call test_polymorphic_parts/test.py::test_polymorphic_parts_non_adaptive 3.45s call test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-True-False] 3.45s call test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-False-False] 3.40s call test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-False-False] 3.40s call test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-True-True] 3.34s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-False-False] 3.34s call test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-True-False] 3.31s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-False-False] 3.29s call test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-False-True] 3.28s teardown test_keeper_snapshot_small_distance/test.py::test_snapshot_and_load 3.25s call test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-False-True] 3.20s call test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-True-True] 3.20s call test_executable_dictionary/test.py::test_executable_source_exit_code_check 3.15s call test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-False-True] 3.15s call test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-True-True] 3.15s call test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-True-False] 3.11s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-False-False] 3.09s call test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-True-True] 3.05s call test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-True-False] 3.04s call test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-False-False] 3.04s call test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-False-False] 3.04s call test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-False-True] 2.99s call test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-True-False] 2.93s call test_polymorphic_parts/test.py::test_polymorphic_parts_index 2.84s call test_grant_and_revoke/test_without_table_engine_grant.py::test_table_engine_and_source_grant 2.78s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-True-False] 2.77s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-True-True] 2.77s call test_executable_dictionary/test.py::test_executable_input_sum_python 2.75s call test_max_authentication_methods_per_user/test.py::test_alter_default_setting 2.75s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-True-False] 2.72s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-True-False] 2.65s call test_max_authentication_methods_per_user/test.py::test_alter 2.65s teardown test_executable_dictionary/test.py::test_executable_source_updated_python 2.52s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-True-True] 2.38s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-False-True] 2.37s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-False-False] 2.32s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-True-True] 2.27s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-True-True] 2.27s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-True-False] 2.22s call test_executable_dictionary/test.py::test_executable_non_direct_input_bash 2.22s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-False-True] 2.22s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-True-True] 2.21s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-False-False] 2.19s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-False-True] 2.17s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-False-False] 2.16s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-True-True] 2.12s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-True-False] 2.12s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-False-False] 2.11s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-True-False] 2.11s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-False-True] 2.07s call test_max_authentication_methods_per_user/test.py::test_create 2.06s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-False-True] 1.96s setup test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-True-False] 1.66s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-False-False] 1.57s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-True-False] 1.53s call test_max_authentication_methods_per_user/test.py::test_create_default_setting 1.45s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-True-True] 1.40s call test_executable_dictionary/test.py::test_executable_input_python 1.39s call test_executable_dictionary/test.py::test_executable_input_send_chunk_header_python 1.38s call test_executable_dictionary/test.py::test_executable_input_signalled_python 1.38s call test_executable_dictionary/test.py::test_executable_input_bash 1.24s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-False-False] 1.20s teardown test_match_process_uid_against_data_owner/test.py::test_different_user 1.09s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-True-True] 1.08s call test_password_constraints/test.py::test_default_password_type 1.02s call test_kerberos_auth/test.py::test_bad_path_to_keytab 0.99s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-False-True] 0.97s call test_kerberos_auth/test.py::test_kerberos_auth_without_keytab 0.93s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-True-True] 0.92s call test_format_avro_confluent/test.py::test_select_auth_encoded 0.87s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-True-False] 0.86s call test_keeper_profiler/test.py::test_profiler 0.85s call test_format_avro_confluent/test.py::test_select_auth_encoded_complex 0.79s call test_kerberos_auth/test.py::test_kerberos_auth_with_keytab 0.64s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-True-False] 0.63s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-False-True] 0.59s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-True-True] 0.58s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-True-False] 0.58s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-True-True] 0.58s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-False-False] 0.58s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-False-True] 0.53s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-False-True] 0.53s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-False-True] 0.51s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-False-False] 0.49s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-True-True] 0.48s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-False-False] 0.48s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-False-True] 0.48s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-False-False] 0.48s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-False-False] 0.48s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-True-False] 0.48s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-True-False] 0.48s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-False-False] 0.48s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-True-False] 0.48s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-False-True] 0.48s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-True-True] 0.48s teardown test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-True-False] 0.40s call test_merge_tree_check_part_with_cache/test.py::test_check_part_with_cache 0.03s teardown test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-1000-SELECT sum(key) FROM {table_name}] 0.02s setup test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-10000-SELECT sum(key) FROM {table_name}] 0.02s setup test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-100000-SELECT sum(key) FROM {table_name}] 0.01s teardown test_polymorphic_parts/test.py::test_compact_parts_only 0.01s setup test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_range-key] 0.01s setup test_polymorphic_parts/test.py::test_different_part_types_on_replicas[polymorphic_table_compact-Compact] 0.01s teardown test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-10000-SELECT sum(key) FROM {table_name}] 0.01s teardown test_log_query_probability/test.py::test_log_quries_probability_one 0.01s teardown test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_range-key] 0.01s setup test_executable_dictionary/test.py::test_executable_source_exit_code_check 0.00s setup test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-1000-SELECT sum(key) FROM {table_name}] 0.00s setup test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_range-sipHash64(key)] 0.00s teardown test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-100000-SELECT sum(key) FROM {table_name}] 0.00s setup test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_sampling-key] 0.00s setup test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-1000-SELECT sum(key) FROM {table_name}] 0.00s teardown test_format_avro_confluent/test.py::test_select 0.00s teardown test_polymorphic_parts/test.py::test_polymorphic_parts_basics[first_node1-second_node1] 0.00s setup test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)] 0.00s teardown test_polymorphic_parts/test.py::test_polymorphic_parts_basics[first_node0-second_node0] 0.00s teardown test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_single_shard_multiple_replicas-custom_key_sampling-key] 0.00s setup test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-1000-SELECT sum(key) FROM {table_name}] 0.00s setup test_polymorphic_parts/test.py::test_polymorphic_parts_basics[first_node1-second_node1] 0.00s setup test_executable_dictionary/test.py::test_executable_input_sum_python 0.00s setup test_log_query_probability/test.py::test_log_quries_probability_two 0.00s teardown test_executable_dictionary/test.py::test_executable_input_slow_python 0.00s teardown test_executable_dictionary/test.py::test_executable_source_argument_python 0.00s setup test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_sampling-key] 0.00s setup test_polymorphic_parts/test.py::test_polymorphic_parts_index 0.00s setup test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)] 0.00s setup test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-100000-SELECT sum(key) FROM {table_name}] 0.00s setup test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_multiple_shards_multiple_replicas-custom_key_sampling-sipHash64(key)] 0.00s teardown test_polymorphic_parts/test.py::test_different_part_types_on_replicas[polymorphic_table_compact-Compact] 0.00s teardown test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-10000-SELECT sum(key) FROM {table_name}] 0.00s teardown test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_range-sipHash64(key)] 0.00s setup test_format_avro_confluent/test.py::test_select_auth 0.00s teardown test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-10000-SELECT sum(key) FROM {table_name}] 0.00s setup test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-100000-SELECT sum(key) FROM {table_name}] 0.00s setup test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)] 0.00s teardown test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-10000-SELECT sum(key) FROM {table_name}] 0.00s setup test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-10000-SELECT sum(key) FROM {table_name}] 0.00s teardown test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_multiple_shards_multiple_replicas-custom_key_sampling-key] 0.00s setup test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-10000-SELECT sum(key) FROM {table_name}] 0.00s teardown test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_sampling-key] 0.00s teardown test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_range-sipHash64(key)] 0.00s setup test_polymorphic_parts/test.py::test_polymorphic_parts_basics[first_node0-second_node0] 0.00s teardown test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-1000-SELECT sum(key) FROM {table_name}] 0.00s setup test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_single_shard_multiple_replicas-custom_key_sampling-key] 0.00s teardown test_format_avro_confluent/test.py::test_select_auth 0.00s teardown test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)] 0.00s teardown test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_sampling-key] 0.00s setup test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-100000-SELECT sum(key) FROM {table_name}] 0.00s setup test_format_avro_confluent/test.py::test_select_auth_encoded_complex 0.00s teardown test_polymorphic_parts/test.py::test_different_part_types_on_replicas[polymorphic_table_wide-Wide] 0.00s teardown test_kerberos_auth/test.py::test_kerberos_auth_with_keytab 0.00s setup test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-10000-SELECT sum(key) FROM {table_name}] 0.00s teardown test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-100000-SELECT sum(key) FROM {table_name}] 0.00s setup test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_range-sipHash64(key)] 0.00s teardown test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-1000-SELECT sum(key) FROM {table_name}] 0.00s teardown test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-100000-SELECT sum(key) FROM {table_name}] 0.00s teardown test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_range-key] 0.00s teardown test_executable_dictionary/test.py::test_executable_source_python 0.00s teardown test_executable_dictionary/test.py::test_executable_input_sum_python 0.00s teardown test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_multiple_shards_multiple_replicas-custom_key_sampling-sipHash64(key)] 0.00s teardown test_kerberos_auth/test.py::test_bad_path_to_keytab 0.00s setup test_polymorphic_parts/test.py::test_different_part_types_on_replicas[polymorphic_table_wide-Wide] 0.00s setup test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_range-key] 0.00s setup test_executable_dictionary/test.py::test_executable_source_updated_python 0.00s setup test_format_avro_confluent/test.py::test_select_auth_encoded 0.00s teardown test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-1000-SELECT sum(key) FROM {table_name}] 0.00s setup test_kerberos_auth/test.py::test_kerberos_auth_without_keytab 0.00s setup test_executable_dictionary/test.py::test_executable_non_direct_input_bash 0.00s setup test_max_authentication_methods_per_user/test.py::test_create_default_setting 0.00s teardown test_format_avro_confluent/test.py::test_select_auth_encoded 0.00s teardown test_password_constraints/test.py::test_complexity_rules 0.00s teardown test_max_authentication_methods_per_user/test.py::test_create 0.00s teardown test_executable_dictionary/test.py::test_executable_input_bash 0.00s teardown test_max_authentication_methods_per_user/test.py::test_alter 0.00s setup test_executable_dictionary/test.py::test_executable_source_python 0.00s teardown test_executable_dictionary/test.py::test_executable_input_python 0.00s setup test_executable_dictionary/test.py::test_executable_source_argument_python 0.00s teardown test_polymorphic_parts/test.py::test_polymorphic_parts_index 0.00s setup test_kerberos_auth/test.py::test_kerberos_auth_with_keytab 0.00s teardown test_executable_dictionary/test.py::test_executable_non_direct_input_bash 0.00s setup test_executable_dictionary/test.py::test_executable_input_slow_python 0.00s setup test_polymorphic_parts/test.py::test_polymorphic_parts_non_adaptive 0.00s teardown test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)] 0.00s setup test_executable_dictionary/test.py::test_executable_input_python 0.00s setup test_max_authentication_methods_per_user/test.py::test_create 0.00s setup test_executable_dictionary/test.py::test_executable_input_signalled_python 0.00s setup test_max_authentication_methods_per_user/test.py::test_alter_default_setting 0.00s setup test_executable_dictionary/test.py::test_executable_input_send_chunk_header_python 0.00s setup test_password_constraints/test.py::test_default_password_type 0.00s teardown test_executable_dictionary/test.py::test_executable_input_send_chunk_header_python 0.00s teardown test_executable_dictionary/test.py::test_executable_input_signalled_python 0.00s teardown test_max_authentication_methods_per_user/test.py::test_alter_default_setting 0.00s teardown test_executable_dictionary/test.py::test_executable_source_exit_code_check =========================== short test summary info ============================ FAILED test_keeper_reconfig_remove_many/test.py::test_reconfig_remove_2_and_leader FAILED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_sampling-key] PASSED test_max_authentication_methods_per_user/test.py::test_alter PASSED test_kerberos_auth/test.py::test_bad_path_to_keytab PASSED test_max_authentication_methods_per_user/test.py::test_alter_default_setting PASSED test_kerberos_auth/test.py::test_kerberos_auth_with_keytab PASSED test_password_constraints/test.py::test_complexity_rules PASSED test_kerberos_auth/test.py::test_kerberos_auth_without_keytab PASSED test_password_constraints/test.py::test_default_password_type PASSED test_max_authentication_methods_per_user/test.py::test_create PASSED test_max_authentication_methods_per_user/test.py::test_create_default_setting PASSED test_executable_dictionary/test.py::test_executable_input_bash PASSED test_executable_dictionary/test.py::test_executable_input_python PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_multiple_shards_multiple_replicas-custom_key_sampling-key] PASSED test_executable_dictionary/test.py::test_executable_input_send_chunk_header_python PASSED test_executable_dictionary/test.py::test_executable_input_signalled_python PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_multiple_shards_multiple_replicas-custom_key_sampling-sipHash64(key)] PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_single_shard_multiple_replicas-custom_key_sampling-key] PASSED test_executable_dictionary/test.py::test_executable_input_slow_python PASSED test_executable_dictionary/test.py::test_executable_input_sum_python PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)] PASSED test_executable_dictionary/test.py::test_executable_non_direct_input_bash PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-False-False] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-1000-SELECT sum(key) FROM {table_name}] PASSED test_executable_dictionary/test.py::test_executable_source_argument_python PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_range-key] PASSED test_executable_dictionary/test.py::test_executable_source_exit_code_check PASSED test_format_avro_confluent/test.py::test_select PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_range-sipHash64(key)] PASSED test_format_avro_confluent/test.py::test_select_auth PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-False-True] PASSED test_executable_dictionary/test.py::test_executable_source_python PASSED test_format_avro_confluent/test.py::test_select_auth_encoded PASSED test_format_avro_confluent/test.py::test_select_auth_encoded_complex PASSED test_groupBitmapAnd_on_distributed/test_groupBitmapAndState_on_distributed_table.py::test_groupBitmapAndState_on_distributed_table PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)] PASSED test_log_query_probability/test.py::test_log_quries_probability_one PASSED test_keeper_profiler/test.py::test_profiler PASSED test_polymorphic_parts/test.py::test_compact_parts_only PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-True-False] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-10000-SELECT sum(key) FROM {table_name}] PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_range-key] PASSED test_polymorphic_parts/test.py::test_different_part_types_on_replicas[polymorphic_table_compact-Compact] PASSED test_executable_dictionary/test.py::test_executable_source_updated_python PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_range-sipHash64(key)] PASSED test_polymorphic_parts/test.py::test_different_part_types_on_replicas[polymorphic_table_wide-Wide] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-True-True] PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_sampling-key] PASSED test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-False-False] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-100000-SELECT sum(key) FROM {table_name}] PASSED test_log_levels_update/test.py::test_log_levels_update PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-False-True] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-1000-SELECT sum(key) FROM {table_name}] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-True-False] PASSED test_profile_events_s3/test.py::test_profile_events PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-True-True] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-10000-SELECT sum(key) FROM {table_name}] PASSED test_polymorphic_parts/test.py::test_polymorphic_parts_basics[first_node0-second_node0] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-False-False] PASSED test_keeper_restore_from_snapshot/test_disk_s3.py::test_recover_from_snapshot_with_disk_s3 PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-False-True] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-100000-SELECT sum(key) FROM {table_name}] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-True-False] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-True-True] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-1000-SELECT sum(key) FROM {table_name}] PASSED test_keeper_snapshot_small_distance/test.py::test_snapshot_and_load PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-False-False] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-10000-SELECT sum(key) FROM {table_name}] PASSED test_polymorphic_parts/test.py::test_polymorphic_parts_basics[first_node1-second_node1] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-False-True] PASSED test_polymorphic_parts/test.py::test_polymorphic_parts_index PASSED test_polymorphic_parts/test.py::test_polymorphic_parts_non_adaptive PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-True-False] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-100000-SELECT sum(key) FROM {table_name}] PASSED test_match_process_uid_against_data_owner/test.py::test_different_user PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-True-True] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-False-False] PASSED test_parallel_replicas_increase_error_count/test.py::test_increase_error_count PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-1000-SELECT sum(key) FROM {table_name}] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-False-True] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-10000-SELECT sum(key) FROM {table_name}] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-True-False] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-True-True] PASSED test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-100000-SELECT sum(key) FROM {table_name}] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-False-False] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-False-True] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-True-False] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-True-True] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-False-False] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-False-True] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-True-False] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-True-True] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-False-False] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-False-True] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-True-False] PASSED test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-True-True] PASSED test_log_query_probability/test.py::test_log_quries_probability_two PASSED test_grant_and_revoke/test_without_table_engine_grant.py::test_table_engine_and_source_grant SKIPPED [1] test_merge_tree_check_part_with_cache/test.py:28: Skip with debug build and sanitizers. This test manually corrupts cache which triggers LOGICAL_ERROR and leads to crash with those builds ============= 2 failed, 97 passed, 1 skipped in 490.98s (0:08:10) ============== Traceback (most recent call last): File "/home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration/./runner", line 492, in subprocess.check_call(cmd, shell=True, bufsize=0) File "/usr/lib/python3.10/subprocess.py", line 369, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command 'docker run --rm --name clickhouse_integration_tests_q1p96k --privileged --dns-search='.' --memory=30709039104 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --volume=/home/ubuntu/_work/_temp/test/build/clickhouse:/clickhouse --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/programs/server:/clickhouse-config --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration:/ClickHouse/tests/integration --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/backupview:/ClickHouse/utils/backupview --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/grpc-client/pb2:/ClickHouse/utils/grpc-client/pb2 --volume=/run:/run/host:ro --volume=clickhouse_integration_tests_volume:/var/lib/docker -e DOCKER_DOTNET_CLIENT_TAG=11de0b29a15d -e DOCKER_HELPER_TAG=5dc43a6382f0 -e DOCKER_BASE_TAG=5ccda723c1fc -e DOCKER_KERBEROS_KDC_TAG=9391ecdee8d7 -e DOCKER_MYSQL_GOLANG_CLIENT_TAG=9bec2a638e6e -e DOCKER_MYSQL_JAVA_CLIENT_TAG=766bff31cfe4 -e DOCKER_MYSQL_JS_CLIENT_TAG=41ba7c2ec2a1 -e DOCKER_MYSQL_PHP_CLIENT_TAG=88be89c1e3b6 -e DOCKER_NGINX_DAV_TAG=b55ac9cd7519 -e DOCKER_POSTGRESQL_JAVA_CLIENT_TAG=a4eff5c7f4d6 -e DOCKER_PYTHON_BOTTLE_TAG=d862517635bf -e DOCKER_CLIENT_TIMEOUT=300 -e COMPOSE_HTTP_TIMEOUT=600 -e CLICKHOUSE_USE_OLD_ANALYZER=1 -e PYTHONUNBUFFERED=1 -e PYTEST_ADDOPTS="--dist=loadfile -n 10 -rfEps --run-id=0 --color=no --durations=0 --report-log=parallel1_0.jsonl --report-log-exclude-logs-on-passed-tests test_executable_dictionary/test.py::test_executable_input_bash test_executable_dictionary/test.py::test_executable_input_python test_executable_dictionary/test.py::test_executable_input_send_chunk_header_python test_executable_dictionary/test.py::test_executable_input_signalled_python test_executable_dictionary/test.py::test_executable_input_slow_python test_executable_dictionary/test.py::test_executable_input_sum_python test_executable_dictionary/test.py::test_executable_non_direct_input_bash test_executable_dictionary/test.py::test_executable_source_argument_python test_executable_dictionary/test.py::test_executable_source_exit_code_check test_executable_dictionary/test.py::test_executable_source_python test_executable_dictionary/test.py::test_executable_source_updated_python test_format_avro_confluent/test.py::test_select test_format_avro_confluent/test.py::test_select_auth test_format_avro_confluent/test.py::test_select_auth_encoded test_format_avro_confluent/test.py::test_select_auth_encoded_complex test_grant_and_revoke/test_without_table_engine_grant.py::test_table_engine_and_source_grant test_groupBitmapAnd_on_distributed/test_groupBitmapAndState_on_distributed_table.py::test_groupBitmapAndState_on_distributed_table test_keeper_profiler/test.py::test_profiler test_keeper_reconfig_remove_many/test.py::test_reconfig_remove_2_and_leader test_keeper_restore_from_snapshot/test_disk_s3.py::test_recover_from_snapshot_with_disk_s3 test_keeper_snapshot_small_distance/test.py::test_snapshot_and_load test_kerberos_auth/test.py::test_bad_path_to_keytab test_kerberos_auth/test.py::test_kerberos_auth_with_keytab test_kerberos_auth/test.py::test_kerberos_auth_without_keytab test_log_levels_update/test.py::test_log_levels_update test_log_query_probability/test.py::test_log_quries_probability_one test_log_query_probability/test.py::test_log_quries_probability_two test_match_process_uid_against_data_owner/test.py::test_different_user test_max_authentication_methods_per_user/test.py::test_alter test_max_authentication_methods_per_user/test.py::test_alter_default_setting test_max_authentication_methods_per_user/test.py::test_create test_max_authentication_methods_per_user/test.py::test_create_default_setting test_merge_tree_check_part_with_cache/test.py::test_check_part_with_cache 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_multiple_shards_multiple_replicas-custom_key_sampling-key]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_multiple_shards_multiple_replicas-custom_key_sampling-sipHash64(key)]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_single_shard_multiple_replicas-custom_key_sampling-key]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_distributed[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_range-key]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_range-sipHash64(key)]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_sampling-key]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_mergetree[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_range-key]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_range-sipHash64(key)]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_sampling-key]' 'test_parallel_replicas_custom_key/test.py::test_parallel_replicas_custom_key_replicatedmergetree[test_single_shard_multiple_replicas-custom_key_sampling-sipHash64(key)]' test_parallel_replicas_increase_error_count/test.py::test_increase_error_count 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-1000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-10000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-101-100000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-1000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-10000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[1-11-100000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-1000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-10000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-101-100000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-1000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-10000-SELECT sum(key) FROM {table_name}]' 'test_parallel_replicas_invisible_parts/test.py::test_reading_with_invisible_parts[11-11-100000-SELECT sum(key) FROM {table_name}]' test_password_constraints/test.py::test_complexity_rules test_password_constraints/test.py::test_default_password_type test_polymorphic_parts/test.py::test_compact_parts_only 'test_polymorphic_parts/test.py::test_different_part_types_on_replicas[polymorphic_table_compact-Compact]' 'test_polymorphic_parts/test.py::test_different_part_types_on_replicas[polymorphic_table_wide-Wide]' 'test_polymorphic_parts/test.py::test_polymorphic_parts_basics[first_node0-second_node0]' 'test_polymorphic_parts/test.py::test_polymorphic_parts_basics[first_node1-second_node1]' test_polymorphic_parts/test.py::test_polymorphic_parts_index test_polymorphic_parts/test.py::test_polymorphic_parts_non_adaptive test_profile_events_s3/test.py::test_profile_events 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-False-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-False-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-True-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-None-True-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-False-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-False-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-True-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-False-depends_on1-True-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-False-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-False-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-True-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-None-True-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-False-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-False-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-True-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings0-test_db-True-depends_on1-True-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-False-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-False-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-True-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-None-True-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-False-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-False-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-True-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-False-depends_on1-True-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-False-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-False-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-True-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-None-True-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-False-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-False-True]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-True-False]' 'test_refreshable_mat_view_replicated/test.py::test_alters[settings1-test_db-True-depends_on1-True-True]' -vvv " altinityinfra/integration-tests-runner:226bfaf75ac1 ' returned non-zero exit status 1.